Collection and processing of sensor readings and other data in typical limited resource systems presents a number of significant implementation problems. Typical limited resource systems have limited memory and processing capabilities. Often, data is collected and produced during interrupt processing on the system, and is subject to very rapid change. In many cases, the timeliness of these changes is critical, and the duration of significant changes may be very short (milliseconds are not uncommon). As a result, a number of serious design problems emerge in such systems for collecting and processing data.
For example, typical systems often have difficulty quickly and efficiently reporting new sensor readings to collection systems. In addition, typical systems are often unable to prevent short-term (millisecond range) updates, which may be significant, from being “missed” by processing code, such as code for implementing value thresholds, or for reporting value updates across a network. Furthermore, certain typical systems are unable to allow “slow running” code, such as notification routines for sending data through e-mail or FTP, to view data in a “stable” state, so that a set of related values can be reported or processed in a consistent state (“temporal integrity”). Also, these systems often have difficulty delivering multiple data values consistently to collection systems, so that processing code does not observe temporary data inconsistencies (“transactional integrity”).
Many typical systems write and rewrite single data structures, typically with some sort of semaphore or other mutual exclusion mechanism, and rely on frequent polling by data monitoring applications to observe problems or report changes. Other systems work on a “process on update” model, where the act of updating the data also includes immediately calling the necessary processing code, so that the data update is handled, but limits how much processing can be done before another data update can be reported and handled.
Large scale data processing systems, such as distributed relational database systems (RDBMS), often face variations of problems, such as frequent update, multiple users with differing rates of processing, and significant integrity requirements. A common mechanism used to aid in addressing these problems in many DBMS systems is the use of a data journal.
Unfortunately, such classic database systems are not suitable for limited resource monitoring systems for a number of reasons. First, they typically require large database applications, along with large amounts of file system based storage. Secondly, their client interfaces are exclusively “user-mode” useable as opposed to being useable by interrupt mode or kernel mode application code. Lastly, typical database systems lack the real-time to near real-time responsiveness needed both for data update and data access by limited resource monitoring system applications. Accordingly, there is a need for an improved system and method of accessing sensor and configuration data.